Mutual feedback in e-portfolio assessment: an approach to the netfolio system

This paper presents an alternative application of e-portfolio in a university student assessment context. A concept based on student collaboration (called netfolio) is developed, that differs from the classical e-portfolio concept. The use of a netfolio, a network of student e-portfolios, in a virtual classroom is explained through an exploratory study. A netfolio is more than a group of e-portfolios because it offers students a better understanding of learning objectives and promotes self-revision through participation in assessment of other students’ learning, as indicated through their portfolios. Class student e-portfolios are interconnected in a unique netfolio such that each student assesses their peers’ work and at the same time is being assessed. This process creates a chain of co-evaluators, facilitating a mutual and progressive improvement process. Results about teachers’ and students’ mutual feedback are presented and the benefits of the process in terms of academic achievements are analysed. The netfolio concept The classic concept of e-portfolio in online education offers great potential for learning, summed up in the extensive and rich snapshot that it provides of the abilities developed by a student in relation to a learning process. In this context, we should stress the reflective aspect of the e-portfolio as one of its undeniable contributions, incorporating in-depth learning that is rich in critical and creative thought (Zubizarreta, 2004). In the cognitive effort of reflecting on the work carried out, the students learn to discuss decisions and relate contents, high-level skills that could otherwise be left in the shade. Another value of the e-portfolio is the continuous improvement that it can offer a student. A student does not see the work as definitive but can steadily improve it over the British Journal of Educational Technology (2008) doi:10.1111/j.1467-8535.2007.00803.x © 2007 The Author. Journal compilation © 2007 British Educational Communications and Technology Agency. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. learning period. Several authors (eg, Cole, Ryan & Kirk, 1995; Hunt & Pellegrino, 2002; Svinicki, 2001) have argued that every learning action should focus on the collaborative construction of knowledge between students and teacher and between the students themselves. In this constructive process, educational help is given to students to acquire a meaningful knowledge (Cambridge, 2001; Riedinger, 2006). One of the leading mechanisms in the educational process is the assessment procedure of learning focused on regular qualitative feedback. Rovai, Ponton, Derrick and Davis (2006) have argued that sometimes the feedback process in online education needs to be more explicit than in face-to-face education to have similar educational effects. The netfolio seems to be a useful tool in achieving this aim because of the inclusion of peer and co-assessment processes and their consequences (Dochy, Segers & Sluijsmans, 1999; Olina & Sullivan, 2004). The concepts of peer assessment and co-assessment may have various different applications in an educational context, and at the same time, as we know, they mean very different things. However, in the case of the netfolio these notions are reincorporated into the framework of profound learning of a collaborative nature. This study does not consider the results of these co-evaluation processes involving both teacher and student in a summative evaluation, as is often the case (Hall, 1995). Instead, they are understood as scaffolding methods at the heart of a formative evaluation directed at improving teaching and learning. Formative evaluation that actively involves students in their development—as is the case with peer and co-assessment, as well as self-assessment—aims to facilitate the student to achieve autonomous and independent learning (Nicol, 2007). This autonomy is achieved through self-regulation processes that come from the application of the mechanisms included in formative evaluation in any of its different forms. In recent years, the use of e-portfolios has played the key role in an effective form of carrying out a formative analysis. The four pillars of the e-portfolio (metacognition, authentic tasks, contextual feedback and student responsibility) seem to clear up some of the principal doubts raised as to the effectiveness of formative evaluation (Black & William, 1998). The focus of the e-portfolio also satisfies those educative models based on exploratory dialogue and guidance as a source of cognitive change (Whitelock, 2006) above all if one opts for an individual model of the learner. Nevertheless, e-portfolios in their most common practice display a number of significant shortcomings. On the one hand, while there are some specific initiatives that consider the social aspect of the e-portfolio (Gordin, Grueneberg, Laff, Martinez & Lam, 2004), their format is most frequently based on the individual creation of work; insofar as the collective value that other students may bring to the work is not taken into account. On the other hand, it is not a complete work because it is not submitted as a single large text but as pieces of work that are very interesting but unconnected to each other. Each shortcoming, the individuality and the disconnection, are factors that the emerging netfolio concept aims to overcome. As the word reflects, the netfolio is con2 British Journal of Educational Technology © 2007 The Author. Journal compilation © 2007 British Educational Communications and Technology Agency. figured through a set of e-portfolios produced by different students who, at a given time and through online communication, provide the other students with new content and different perspectives. This collaborative approach in the form of a network offers students the ability to compare e-portfolios as another stage in the construction of their own personal e-portfolio. In short, the netfolio overcomes the limitation of working alone and the restricted range of learning experiences that characterise ‘classic’ e-portfolios. For its part, a netfolio is basically a large mesh of different learning outcomes driven by two assumptions: (1) It offers different interpretations of each piece of students’ work corresponding to a learning objective or a specific competence in terms of professional skills put forward by diverse students; and (2) it understands each student’s portfolio as a multi-text that interweaves the different demonstrations of learning outcomes (called hereafter ‘evidence’ of learning as examples of student work that reveals the level of accomplishment of a competence) into a larger text with a global and interrelated meaning. The first assumption refers to the communication between students that validates or clarifies the demonstration of an agreed core of competence. The second assumption refers to the improved connections and interlinking resulting from the integration of a student’s own demonstration of learning outcomes with those of other students. Together, these two assumptions, external and internal, weave a solid network of evidences of learning and make an additional contribution based on the different inputs, both for the teacher and for the students themselves. As well as examples of progress, the teacher is presented with an alternative and supplement to their own teaching in the sense that it is the other students who show examples of their work and act as instructors by adding their own work to that of the others. The student is not only afforded this gallery of different works provided by other students for comparison, adjustment and improvement of their own productions, they are also forced into a complex (re)production of their own work. The netfolio concept does not replace that of the e-portfolio or even extends its role, but rather draws attention to a more complete teaching and learning system. Very often, the instructional design of e-portfolios is limited by the chosen technology (Acker, 2005; Ehrmann, 2002). It is therefore a question, again and again in each case, of proposing the competences that are to be developed in the students and of attempting to reinvent and adapt the technology to carry out what is pedagogically required, through solid initial educational design (Ehrmann, 2006). On some occasions, the technology interferes with the educational intentions, taking them for granted; as a result, we find initial ideas that are good but prove incomplete when transferred to information technology (IT) systems. Nevertheless, IT systems in themselves do have the potential of adapting to almost any educational requirement. Architecture of the netfolio As an exploratory study, the netfolio concept has been applied to a PhD course on the Information and Knowledge Society at a virtual university that has been running all of Mutual feedback in e-portfolio 3 © 2007 The Author. Journal compilation © 2007 British Educational Communications and Technology Agency. its courses completely online from scratch since 1995. The objectives of this course are to develop basic research competences within the framework of online education. The competences are organised in phases: defining the research question, making hypotheses, finding information and the presentation of results, integration of research results, comparison of hypotheses and drafting of conclusions. The netfolio has been integrated into the university’s e-learning platform and has been designed technically and pedagogically to provide learning support to students online in such a way that they receive feedback at every phase for the improvement of their work. In this case, the netfolio was structured in three sections visible to the students at all times. These were: ‘presentation’, ‘competences’ and ‘monitoring’. A classic learningtype e-portfolio could have these three parts but, as we have seen, in the case of the netfolio the internal dimensions have been strengthened with regard to: (1) relations between evidences of learning (good student work examples) that show the achievement of a specific competence—students have the possibility to consult all the evidences of the others using the same IT platform through specific links marked by a special forum icon to this effect; and (2) relations between evidences of learning in the same e-portfolio—a single multi-text made up of a student’s different evidences is created, through an explanatory forum that integrates all the phases completed in parts. Although this integration is located in a specific part of the introductory electronic page to make sense of the whole e-portfolio from the beginning (presentation section), it is only completed once the e-portfolio is finished. In this way, in the netfolio, each student has and benefits from two different perspectives on the work of others: one is partial, centred on each of the skills and the other more global, in which the student reclaims the central role of a subject learning a discipline. At a technological level, both perspectives are shared by students through specific forums. The first is located in a forum site in each competence section, to which all students have access; the other emerges in the student’s personal forum, which can be accessed by the other students. The final result of the contributions to the latter forum is what should be reflected in the introductory page (presentation section) of the netfolio as cognitive changes induced by equals. The ‘presentation’, ‘competences’ and ‘monitoring’ sections are laid out in three horizontal tabs, which configure the main menu of the netfolio. In detail, ‘presentation’ includes in the foreground a personal motto and a photo, both of which are related to the student but necessarily also to the specific contents of the course—because they have their general biography in another part of the virtual campus. A global introduction of the whole portfolio that explains the integration of comments about the evidences of learning mentioned previously is also included in this section. Complementarily, the best works produced as professional and/or student and the approach are shown on the same page, and optionally the professional milestones. The ‘competences’ section includes the drafting of the competences and the evidence required, and displays the part of the rubric corresponding to the competence selected. 4 British Journal of Educational Technology © 2007 The Author. Journal compilation © 2007 British Educational Communications and Technology Agency. Once a competence has been selected, an exclusive place (dialogue page) is provided to attach it, justify it, reflect on it and self-assess it. An important part of this space is that each evidence has a private discussion forum. This forum is aimed at the improvement of the evidence itself and the competence related to the evidence, which is always open to the comments of the teacher and, at the time of its presentation, also open to the opinions and assessments of other students on the course. In the ‘monitoring’ section, the complete rubric of the course is displayed in which three types of qualitative assessments appear in an automatic box: (1) Self-assessments of the students themselves; (2) assessments of other students in the e-portfolio (mutual assessments); and (3) the teacher’s assessments. The student has the possibility of writing personal comments answering the teacher’s comments on his or her work in a specific discussion area within the same page. In this section the teacher views all of the portfolios and can make the different portfolios visible to other students according to the competences that he or she decides, when he or she thinks it necessary or at the request of the students themselves. In this section, each student has a summary box of their work displayed in such a way that they can link them and relate them in any way they see fit. They also have the synthesis of their assessments and an automatic alert system that informs them of the completion status of each of the evidences corresponding to each competence (not started, in progress, in revision, being modified and closed). Although it is important to understand the netfolio structure, which is largely shared with the e-portfolio, it is more important to understand the nature of the interaction between students and with the teacher. This is where the main differences between the two types of portfolio lie—in the communication exchanges taking place in the different forums (referred to in previous discussion) that have been added to the netfolio (in the absence of more adequate technological solutions). These forums store the mutual feedback comments that solidify the students’ social or learning network. These dialogues will later be subject to analysis, in order to verify the netfolio’s real contributions to the methodological mechanism of progressive improvement in relation to cognitive change. Contributions of the netfolio: empirical work